Ultrasound imaging system with automatic image saving

ABSTRACT

Ultrasound imaging systems for automatically identifying and saving ultrasound images relevant to a needle injection procedure, and associated systems and methods, are described herein. For example, an ultrasound imaging system includes a transducer for transmitting/receiving ultrasound signals during a needle injection procedure, and receive circuitry configured to convert the received ultrasound signals into ultrasound image data. The image data can be stored in a buffer memory. A processor can analyze the image data stored in the buffer memory to identify image data that depicts a specified injection event of the needle injection procedure, and the identified image data can be stored in a memory for archival purposes.

TECHNICAL FIELD

The disclosed technology relates to ultrasound imaging systems, and inparticular to systems for improving workflow within clinical settingsusing ultrasound imaging systems.

BACKGROUND

In ultrasound imaging, an operator of a system uses a transducer probeto obtain ultrasound images of a patient during an examination. Theimages captured by the system may be viewed, printed, and/or included ina patient report for diagnosis and record keeping. In addition, selectimages may be included in a written and/or electronic report that isused to bill the patient or their insurance for the services rendered.Depending on the examination procedure being performed, the number andsubject of the images required in a report of the examination may bestandardized or defined. For example, a needle injection procedure, suchas an ultrasound-guided regional anesthesia injection, may require animage of the needle at a target location, an image of the needle duringthe injection, etc.

In a typical single operator examination, a physician or an ultrasoundtechnician uses the imaging system to obtain all the images needed tocomplete the examination. However, during some needle procedures, wherethe care provider cannot stop mid procedure or has no free hands tocontrol the system, a second person may assist in controlling the systemsettings and collecting the needed images during the procedure. Theobtained images are typically stored in a buffer memory and must bereviewed after the examination is complete to mark or otherwise identifyimages to be used in creating a record of the examination. Often,clinicians collect a loop of images (“clip”) and review the loop afterthe examination is complete to select images to be used in creating arecord of the examination. This additional step requires extra time. Ifsingle images are collected instead of a loop, it can be difficult forthe operator to know at the time of the procedure whether the requiredimages of the examination have been captured.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified illustration of an ultrasound imaging system inaccordance with an embodiment of the present technology.

FIG. 2 is a block diagram of the ultrasound imaging system shown in FIG.1 in accordance with an embodiment of the present technology.

FIG. 3 is a flow diagram of a method or process of identifying imageframes stored in a buffer memory that depict an injection event inaccordance with an embodiment of the present technology.

FIG. 4 is a flow diagram of a method or process of identifying imageframes stored in a buffer memory that depict an injection event inaccordance with another embodiment of the present technology.

DETAILED DESCRIPTION

Specific details of several embodiments of ultrasound systems forautomatically saving ultrasound images generated during an ultrasoundimaging procedure using an interventional device, and associated devicesand methods, are described below reference to FIGS. 1-4 . In someembodiments, for example, an ultrasound imaging system includes atransducer configured to transmit ultrasound signals to and receiveultrasound signals from a region of interest during an ultrasound-guidedneedle injection procedure. The ultrasound imaging system furtherincludes receive circuitry configured to convert the received ultrasoundsignals into image frames of ultrasound data, and a buffer memory inwhich the image frames are stored. The ultrasound imaging system alsoincludes a processor configured to analyze the image frames stored inthe buffer memory and identify and mark one or more of the image framesthat depict an event of the needle injection procedure—for example, thedelivery of a fluid from the needle used during the procedure. In someembodiments, the processor can further be configured to save the imageframes depicting the injection event in a memory—other than the buffermemory—for archival purposes.

Although many of the embodiments described below are described withrespect to devices, systems, and methods for automatically saving anultrasound image during a needle injection procedure in which a needleis used to deliver anesthesia or other drugs to a desired location,other applications and other embodiments in addition to those describedherein are within the scope of the present technology. For example, atleast some embodiments of the present technology may be useful forprocedures employing other invasive medical instruments. It should benoted that other embodiments in addition to those disclosed herein arewithin the scope of the present technology.

Further, embodiments of the present technology can have differentconfigurations, components, and/or procedures than those shown ordescribed herein. Moreover, a person of ordinary skill in the art willunderstand that embodiments of the present technology can haveconfigurations, components, and/or procedures in addition to those shownor described herein and that these and other embodiments can be withoutseveral of the configurations, components, and/or procedures shown ordescribed herein without deviating from the present technology. Thephrases “in some embodiments,” “according to some embodiments,” “incertain embodiments,” “in the illustrated embodiment,” “in otherembodiments,” and the like generally mean the particular feature,structure, or characteristic following the phrase is included in atleast one implementation of the present technology, and may be includedin more than one implementation. In addition, such phrases do notnecessarily refer to the same embodiments or different embodiments.

The terminology used below is to be interpreted in its broadestreasonable manner, even though it is being used in conjunction with adetailed description of certain examples of embodiments of thetechnology. Indeed, certain terms may even be emphasized below; however,any terminology intended to be interpreted in any restricted manner willbe overtly and specifically defined as such in this Detailed Descriptionsection.

FIG. 1 shows a representative ultrasound imaging system 100 (“system100”) that implements the present technology for imaging the tissue of asubject or patient 1. In one embodiment, the system 100 can be ahand-held, portable or cart-based system that uses a transducer probe102 to transmit ultrasound signals into a region of interest and toreceive the corresponding echo signals in order to produce an image ofthe tissue being scanned. The transducer probe 102 can be a one or twodimensional linear or curved transducer, a phased array transducer, oranother type of ultrasound transducer as is well known in the art. Thesystem 100 converts characteristics of the received echo signals (e.g.,their amplitude, phase, power, frequency shift, etc.) into data that isquantified and displayed for the user as an image on a video monitor,screen, or other display 104 (“display 104”). As described in detailbelow, the images created may also be stored electronically for digitalrecord keeping or transmitted via a wired or wireless communication linkto another device or location.

In some embodiments, the system 100 can be used during a needleinjection procedure in which an operator of the system 100 guides aninterventional instrument such as a needle 3 into the patient 1 with onehand while holding the transducer probe 102 with the other hand. Incertain embodiments, the operator can view a composite image 106 of thetissue and a representation 107 of where the needle 3 is located in thetissue. The composite image 106 can be updated on the display 104 whilethe needle 3 is guided to a target location within the patient 1. Thetarget location may be a particular nerve site (e.g., when the needleinjection procedure is a regional anesthesia procedure) or other area ofinterest such as a vessel or a particular organ (e.g., uterus, prostate,tumor, heart vessel etc.). In some embodiments, both the needle 3 and aninjectate (e.g., a drug) delivered via the needle 3 can be seen in thecomposite image 106. For example, if the injection is made in a placewith low resistance (e.g., appearing dark in the composite image 106),the injectate can fill the space such that characteristic fluid motionis visible in the composite image 106. In some embodiments, a materialremoved by the needle 3 (e.g., during a biopsy procedure) can be seen inthe composite image 106.

FIG. 2 is a simplified block diagram of the system 100 configured inaccordance with an embodiment of the present technology. As will beappreciated by those skilled in the art, the system 100 may beconstructed with components that are different than those shown in FIG.2 . In addition, the system 100 can include components that are notdiscussed (e.g., a power supply, etc.) and that are not necessary forthe understanding of how to make and use the present technology.

In the illustrated embodiment, the transducer probe 102 is connected toa high voltage multiplexer/de-multiplexer (HV mux/demux) 208 that isused select individual or groups of transducer elements in thetransducer probe 102. Signals to be transmitted by the transducer probe102 are generated by a transmit (TX) beamformer 210 that adjusts thetiming of the signals in order to direct the signals in a particulardirection and to focus the signals at a particular depth in the tissue.Alternatively, unfocused (plane) waves can be transmitted by thetransducer probe 102. Signals from the TX beamformer 210 are amplifiedby one or more high-voltage amplifiers (HV amps) 212 before beingapplied to the HV mux/demux 208 and the transducer probe 102. In otherembodiments, however, signals from the TX beamformer 210 can be passedto the transducer probe 102 directly without an interveningmultiplexer/demultiplexer.

A transmit/receive (T/R) switch 214 operates to disconnect the receiveelectronics of the system 100 from the transducer probe 102 when thehigher powered transmit pulses are being transmitted. The T/R switch 214is closed when the system 100 is to detect the returning echo signals.Signals received by the T/R switch 214 are amplified by low-noisereceive amplifiers (RX amps) 216 that implement a gain function thattypically varies according to the depth from which the echo signalsoriginate. Where the system 100 is a directional ultrasound system, theoutputs of the RX amps 216 feed a receive (RX) beamformer 218 thatdelays and sums the amplified received echo signals. In someembodiments, the analog received signals are converted to correspondingdigital signals, after amplification, with a number of analog to digitalconverters (not shown) that are positioned in the signal path betweenthe RX amps 216 and the RX beamformer 218.

In some embodiments, a system processor 220, which can be implemented asone or more programmed microprocessors, is configured to execute programinstructions that are stored in an internal or external computerreadable memory (not shown) to control the operation of the system 100.As further illustrated in FIG. 2 , beamformed ultrasound signalsproduced by the RX beamformer 218 are delivered to an image processor222. The image processor 222, which may include one or more generalpurpose microprocessors (including the system processor 220), one ormore digital signal processors (DSP), one or more graphics processorunits (GPU), application-specific integrated circuits (ASIC) or thelike, converts the raw, beamformed signals into a two-dimensional imageframe of pixel data that can be stored in memory and shown to theoperator on the display 104.

The image frames produced by the image processor 222 are stored in abuffer memory 224 (also known as a cine buffer), which in one embodimentis operated as a circular buffer of memory elements that stores a selectnumber of image frames as they are produced during an ultrasound imagingprocedure using the system 100. The image frames can be captured usingeither a retrospective or prospective capture mode, as is known in theart. In one embodiment, the buffer memory 224 can store 2-5 minutes ofdata or 3600-9000 image frames of ultrasound data or more. In oneembodiment, once the buffer memory 224 is full, the oldest image framestored in the buffer memory 224 is overwritten with a new image frame ina circular fashion. In the illustrated embodiment, a memory 228 is usedto store the image frames for archival purposes. The contents of thememory 228 may be transferred to a remote patient records keeping systemafter an imaging procedure is complete. In some embodiments, at leastsome of the image frames that are stored in the memory 228 arecompressed to save space and therefore may lack some detail comparedwith the image frames that are stored in the buffer memory 224. In someembodiments, image data other than image frames (e.g., raw image data,pre-conversion image data, etc.) can be stored in the buffer memory 224.

In the illustrated embodiment, the system 100 includes a number ofoperator inputs 230 such as keys, buttons, knobs, a microphone toreceive voice commands, a camera to capture gestures, orsoftware-configured controls, such as touch screen controls or the like.The operator inputs 230 allow an operator to change the operatingcharacteristics of the system 100 and to input commands to the systemprocessor 220.

In some embodiments, the operator begins an ultrasound imaging procedure(e.g., an examination) by using the operator inputs 230 to select aprocedure type from a number of pre-defined procedure types that areshown on the display 104 or that may have a dedicated control on akeyboard or other input device of the system 100. For example, theimaging procedure could be a regional anesthesia injection or otherultrasound-guided needle injection procedure. Each procedure type may beassociated with particular views and/or measurements that are to becaptured by the operator during the specific procedure. For example,some needle injection procedures may require an image of the needle atthe target region with the patient, an image of the needle duringinjection of a drug or other injectate, etc. For example, a nerve blockprocedure may require that three or more different image frames berecorded including views of (i) a needle approaching a target nerve,(ii) the needle at the position of the target nerve, and (iii)anesthetic being delivered around the target nerve. Such views may berequired to be stored (e.g., archived) by the medical facility oroperator for insurance billing purposes. In the illustrated embodiment,the views and/or measurements required by the various procedure typesare stored in a knowledge base 232 (e.g., a memory, database, etc.) thatis accessible to the system processor 220. In some embodiments, theknowledge base 232 can further store one or more parameters, imageframes, or other data from previous examinations that can be, forexample, compared to the image frames stored in the buffer memory 224 asdescribed in detail below.

After selecting a particular imaging procedure, the operator can use oneor more of the operator inputs 230 (e.g., an on-screen button,footswitch, control on an imaging probe, etc.) to begin capturingultrasound image frames using the system 100. These image frames areproduced and stored in the buffer memory 224 until the operator uses oneor more of the operator inputs 230 to halt the image capture process. Insome embodiments, the image capture process can be halted based on othercriteria such as, for example, a timer, an electrocardiogram signal,etc. As will be appreciated, the buffer memory 224 can be constructed tostore several thousand image frames of ultrasound data.

In the past, using conventional ultrasound systems, the operator wasrequired to take the time to review/search through all of the storedimage frames to select which frames would be included in a patient'srecord and/or submitted for billing purposes. Moreover, thereview/search for relevant images was conducted after the imagingprocedure concluded—meaning it was not possible for the operator to knowwith certainty during a needle injection procedure whether they capturedthe required views (e.g., for billing purposes). In some otherconventional ultrasound systems, the operator could press a button on anultrasound probe or use a voice control during a procedure to print orsave particular images—for example, those taken at approximately thetime of an injection. While such systems may reduce the amount of reviewrequired by the operator to identify relevant image frames, buttons maybe pressed accidentally, and voice controls may be accidentallyactivated in noisy environments. Furthermore, it is often cumbersome forthe operator to conduct a needle injection procedure whilesimultaneously triggering an ultrasound system to save certain imagesduring the procedure.

In contrast to conventional systems, the system 100 of the presenttechnology is configured to automatically identify (e.g., select,determine, etc.) and save image frames that are relevant to, orrequired, by a needle injection procedure—for example, those imageframes depicting a particular event, trigger, or aspect of the needleinjection procedure (referred to herein as an “injection event”). Forexample, the system 100 can be configured to automatically identifyimage frames that depict a specified injection event such as a needlebeing positioned at or approaching a target location, an injectiontaking place, etc. Specifically, the system processor 220, the imageprocessor 222, and/or another suitable processor such as a DSP or a GPUcan execute a number of programmed instructions to analyze the imageframes that are stored in the buffer memory 224 and identify one or moreof the image frames that likely depict the specified injection event. Inother embodiments, image data other than image frames (e.g.,pre-conversion image data) can be analyzed to identify portions of theimage data that depict or relate to the specified injection event.

The identified image frames can be saved to the memory 228 for archivalor other purposes. In some embodiments, the identified image frames arestored in the memory 228 with lossless compression or with little or nocompression compared to the unselected image frames generated during theexamination in order to retain more image detail. In certainembodiments, the identified image frames are marked for inclusion in, orautomatically entered into, a patient report of an examination. In someembodiments, the system 100 can generate an indication (e.g., a sound,display, etc.) to provide real-time or near-real-time feedback to theoperator that image frames showing the specified injection event weresuccessfully captured.

The system processor 220 (or another processor) can automaticallyidentify image frames stored in the buffer memory 224 that likely depicta specified injection event in a number of different manners. FIG. 3 ,for example, is a flow diagram of a process or method 300 performed byone or more of the processors in the system 100 for identifying imageframes that depict an injection event using frame-to-frame comparison inaccordance with embodiments of the present technology. Beginning atblock 302, the method 300 includes generating ultrasound signals andreceiving the corresponding echo signals for a needle injectionprocedure and storing image frames of ultrasound data about theprocedure in the buffer memory 224, as described in detail above. Atblock 304, the method includes comparing and/or analyzing image framesstored in the buffer memory 224 to identify the image frames that depictthe specified injection event. In some embodiments, for example, thesystem processor 220 correlates and/or compares the image frames storedin the buffer memory 224 to detect changes between the image frames suchas those caused by motion of a needle and/or the fluid injected from theneedle.

For example, the system processor 220 can use well-known imageprocessing methods to estimate and characterize the point correspondencefrom one image frame to the next in the vicinity of the needle. Forexample, the system processor 220 can estimate the correspondence (ormotion) of each point of interest by maximizing a figure of merit forthe match between a patch of one image frame centered about the point inquestion with a sliding window in the next image frame for each imageframe stored in the buffer memory 224. That is, the system processor 220can estimate flow vectors for each or a subset of pixels between two ormore image frames. Potential figures of merit include 2D correlation,mutual information, or structural similarity. In some embodiments,regularization or pre-processing may be applied to assist in (e.g.,lower the processing costs of) estimating the correspondence of eachpoint of interest. In other embodiments, optical flow methods such asthe Lucas—Kanade or Horn-Schunck may be employed to establish the pointcorrespondences (e.g., flow vectors). In either case, the flow vectorproperties can be classified to indicate the needle and/or injectate.

Regardless of the specific image processing method(s) employed, thesystem processor 220 can identify that certain image frames stored inthe buffer memory 224 likely depict the specified injection event basedon the estimated optical flow/motion between the image frames. In someembodiments, for example, the system processor 220 can determine thatthe needle is stationary or generally stationary in certain image framesand thus that the needle is positioned at the target location and/orthat an injection is occurring in those image frames. Similarly, adetection of fluid motion around the needle tip in certain image framescan indicate that the injection is depicted in those frames.

In other embodiments, the system processor 220 may execute instructionsthat implement a trained neural network or a machine learning algorithmto analyze/compare image frames stored in the buffer memory 224 in orderto identify the image frames that depict the specified injection event.The machine learning algorithm can be an artificial neural network ordeep learning algorithm that is trained to recognize motion indicativeof the injection event in the image frames (e.g., the swirl or expansionof the fluid of the injection around the tip of the needle). In someembodiments, for example, a neural network can be used to detect theinjection event based on the differences between image frames in thebuffer memory 224 (e.g., based on a time series of image frames). Insome embodiments, the machine learning algorithm can be trained basedone or more image frames that were generated and saved in one or moreprevious examinations and that are similar to the image frames requiredto be saved in the current examination. These previous images can bestored in, for example, the knowledge base 232. Based on the previousimages, the machine learning algorithm can determine the image frames inthe buffer memory 224 that, for example, bear the closest resemblance tothe previous images and thus likely depict the injection event. In thismanner, the system processor 220 can identify the image frames thatlikely depict the injection event based on the historical data/examplesfrom previous examinations.

In certain embodiments, the system processor 220 can further mark oridentify a particular region (e.g., a sub-region) in one or more of theimage frames that likely depicts the injection. For example, the systemprocessor 220 could automatically determine a bounding box around thelikely region of injection based on classification of flow vectorscalculated or determined for the image frame(s).

In some embodiments, the system processor 220 can be configured toanalyze only a portion (e.g., a specific region) of the image framesand/or a subset of the total number of image frames in order to detectthe specified injection event. For example, the system processor 220 canbe configured analyze only a portion (e.g., a region) of the imageframes that is proximate to a determined location of the needle.Likewise, the system processor 220 could employ different levels ortypes of imaging processing methods on different subsets of the imageframes stored in the buffer memory 224. For example, the systemprocessor could analyze the image frames stored in the buffer memory 224until detecting that the needle has stopped moving, and then analyzesubsequently generated/stored image frames to detect those image framesthat depict the fluid from the injection around the needle. Suchembodiments can advantageously reduce the processing burden of thesystem 100 by localizing some of the imaging processing steps to asubset of the image frames stored in the buffer memory 224.

At bock 306, the method 300 includes automatically storing the imageframes identified as depicting the injection event in a memory forarchival purposes (e.g., for inclusion in a patient report, billingrecord, or other report). For example, the image frames can be stored inthe memory 228 and/or in a different memory accessible to the systemprocessor 220. In some embodiments, the identified image frames can bemarked or flagged in the buffer memory 224 in addition to or instead ofbeing stored in a memory for archival purposes.

In some embodiments, the system processor 220 can automatically identifyimage frames stored in the buffer memory 224 in other manners (e.g.,other than by a frame-to-frame comparison). FIG. 4 , for example, is aflow diagram of a process or method 400 performed by one or more of theprocessors in the system 100 for identifying image frames that depict aninjection event based on flow information in the image frames, inaccordance with embodiments of the present technology. Beginning atblock 402, the method 400 includes generating ultrasound signals andreceiving the corresponding echo signals for a needle injectionprocedure and storing image frames of ultrasound data about theprocedure in the buffer memory 224, as described in detail above. Morespecifically, in some embodiments, the system 100 can be operated in aDoppler Mode or another color flow imaging mode that generallycorrelates multiple ultrasound signal pulses into a single image framethat contains motion or flow information (often displayed in color), asis known in the art. In some embodiments, the color flow imaging neednot employ the entire color flow processing chain—instead segmentationand classification of a subset of the signals (e.g., lag-0 correlations,lag-1 correlations, etc.) can be used to produce the flow information.At block 304, the system processor 220 can proceed to analyze the flowinformation of the image frames stored in the buffer memory 224 toidentify those image frames depicting the specified injection event.

In some embodiments, the method 400 requires a relatively greater amountof front-end processing by the processor(s) than the method 300illustrated in FIG. 3 , because multiple ultrasound signals pulses areused to generate each image frame. Accordingly, the frame rate of thesystem 100 can be slower when performing the method 400 as compared tothe method 300. In certain embodiments, to improve the frame rate of thesystem 100 while performing the method 400, the system 100 can beconfigured to capture flow information (e.g., to generate, detect, andcorrelate multiple ultrasound signal pulses) for only a portion orregion of each image frame. In some embodiments, for example, the system100 can determine a bounding box (e.g., a virtual color box) or otherregion in which to search for the specified injection event that issmaller than the entire image frame. For example, the system processor220 can execute instructions that implement an image classificationalgorithm (e.g., a detection, trained neural network, or machinelearning algorithm) to detect the location of the needle (e.g., bydetecting a bright reflector, either stationary or moving) and/or thelocation of the needle tip (e.g., by detecting a transition from abright reflector). The system processor 200 can then specify a smallregion around the needle in which to search for an injection of fluidfrom the needle by generating flow information for the small region. Insome embodiments, the detected location of the needle in one image framecan be used to expedite the search for the needle in another image frame(e.g., a subsequent image frame). That is, once the needle is detectedin one image frame, the entirety of each subsequent image frame does notneed to be classified from there forward in time. Finally, at block 406,the method 400 includes automatically storing the image framesidentified as depicting the injection event in a memory for archivalpurposes, or otherwise marking or flagging the image frames in thebuffer memory 224.

In general, the system 100 can automatically identify image framesstored in the buffer memory 224 that likely depict the specifiedinjection event using any of frame-to-frame image processing, imageprocessing using trained neural networks or machine learning algorithms,optical flow (e.g., classification of flow vectors that indicate thecessation of movement of a needle structure and/or the swirling orexpanding of a flow pattern of injectate), color flow imaging, orcombinations thereof. For example, simple frame-to-frame comparisonmethods could be used to detect motion of the needle between imageframes, and specifically to detect that motion of the needle hasstopped. A trained neural network or machine learning algorithm couldthen be employed to detect the needle tip in subsequently generatedimage frames. Finally, a region around the identified needle tip couldbe interrogated using color flow imaging to detect the injection offluid from the needle tip. In this manner, the system 100 canautomatically accurately identify image frames depicting the specifiedinjection event while reducing the processing requirements of the system100 compared to, for example, generating flow information for each fullimage frame produced during a procedure.

In some embodiments, the system 100 is configured to analyze the imageframes stored in the buffer memory 224 in real-time or near-real-time asthey are generated and added to the buffer memory 224. In suchembodiments, after identifying that one or more of the image framesdepicts the specified injection event, the system 100 can be configuredto generate a notification or indication that at least one of the imageframes depicts the injection event. In some embodiments, for example,the system processor 220 can cause a notification to be displayed on thedisplay 226 or a sound to be played via a speaker of the system 100.Accordingly, the system 100 can advantageously provide feedback alertingthe operator of the system 100—during the examination—that the necessaryimage frames were captured. In other embodiments, image frames from anentire ultrasound procedure can be accessed and analyzed after theprocedure is concluded to automatically determine and save image framesdepicting the specified injection event.

In some embodiments, the image frames are associated with meta-data(e.g., narrative information) about the image frame. The meta-data mayinclude the type of tissue being imaged and one or more parameters ofthe system 100 used to obtain the image. Such parameters can include theoperating mode of the ultrasound machine (B-mode, Doppler Mode, Powermode, etc.) as well as power settings, pulse repetition rate, focaldepth, probe used, likelihood of needle presence/motion, etc. In someembodiments, the system processor 220 can insert one or more of theidentified image frames into a patient record (e.g., PACS system,knowledge repository, etc.), patient chart, or other record, while alsoinserting (e.g., pre-populating) the corresponding operating parametersof the ultrasound machine that were used to obtain the image frameand/or other meta-data so that the operator does not have to enter themmanually.

Embodiments of the subject matter and the operations described in thisspecification can be implemented in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Embodiments of the subject matterdescribed in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus.

A computer storage medium can be, or can be included in, acomputer-readable storage device, a computer-readable storage substrate,a random or serial access memory array or device, or a combination ofone or more of them. Moreover, while a computer storage medium is not apropagated signal, a computer storage medium can be a source ordestination of computer program instructions encoded in anartificially-generated propagated signal. The computer storage mediumalso can be, or can be included in, one or more separate physicalcomponents or media (e.g., multiple CDs, disks, or other storagedevices). The operations described in this specification can beimplemented as operations performed by a data processing apparatus ondata stored on one or more computer-readable storage devices or receivedfrom other sources.

The term “processor” encompasses all kinds of apparatus, devices, andmachines for processing data, including by way of example a programmableprocessor, a computer, a system on a chip, or multiple ones, orcombinations, of the foregoing. The apparatus can include specialpurpose logic circuitry, e.g., an FPGA (field programmable gate array)or an ASIC (application-specific integrated circuit). The apparatus alsocan include, in addition to hardware, code that creates an executionenvironment for the computer program in question, e.g., code thatconstitutes processor firmware, a protocol stack, a database managementsystem, an operating system, a cross-platform runtime environment, avirtual machine, or a combination of one or more of them. The apparatusand execution environment can realize various different computing modelinfrastructures, such as web services, distributed computing and gridcomputing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto—optical disks, or optical disks.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a imagingsystem having a display device, e.g., an LCD (liquid crystal display),LED (light emitting diode), or OLED (organic light emitting diode)monitor, for displaying information to the operator and a keyboard and apointing device, e.g., a mouse or a trackball, by which the operator canprovide input to the computer. In some implementations, a touch screencan be used to display information and to receive input from a user.Other kinds of devices can be used to provide for interaction with anoperator as well; for example, feedback provided to the operator can beany form of sensory feedback, e.g., visual feedback, auditory feedback,or tactile feedback; and input from the operator can be received in anyform, including acoustic, speech, or tactile input. In addition, acomputer can interact with an operator by sending documents to andreceiving documents from a device that is used by the user; for example,by sending web pages to a web browser on a user's client device inresponse to requests received from the web browser.

From the foregoing, it will be appreciated that specific embodiments ofthe technology have been described herein for purposes of illustration,but that various modifications may be made without deviating from thedisclosure. Accordingly, the invention is not limited except as by theappended claims. Furthermore, certain aspects of the new technologydescribed in the context of particular embodiments may also be combinedor eliminated in other embodiments. Moreover, although advantagesassociated with certain embodiments of the new technology have beendescribed in the context of those embodiments, other embodiments mayalso exhibit such advantages and not all embodiments need necessarilyexhibit such advantages to fall within the scope of the technology.Accordingly, the disclosure and associated technology can encompassother embodiments not expressly shown or described herein.

We claim:
 1. An ultrasound imaging system, comprising: a transducerconfigured to transmit ultrasound signals to and receive echo signalsfrom a region of interest during a needle injection procedure; a receivecircuitry configured to convert the echo signals into ultrasound imagedata; a buffer memory to store image frames of the ultrasound imagedata; and a processor; a first memory coupled to the processor, whereinthe processor is configured to: obtain a first subset of the imageframes from the buffer memory; compare the first subset of the imageframes to each other, identify, from the first subset, a first imageframe indicating that a needle used for a needle injection procedure isapproaching to a target and a second image frame indicating that theneedle has stopped moving at the target based on the comparing; obtain asecond subset of the image frames from the buffer memory, wherein thesecond subset of the image frames have been generated and storedsubsequent to the first subset in the buffer memory; identify, from thesecond subset, a third image from indicating a tip of the needle using amachine learning algorithm; obtain a third subset of the image framesdepicting a region around the tip of the needle from the buffer memory,wherein the third subset of the image frames have been generated andstored subsequent to the second subset in the buffer memory; identify,from the third subset, a fourth image frame indicating an injection of afluid from the tip of the needle; and save at least three of the firstimage frame, the second image frame, the third image frame and thefourth image frame in the first memory to reduce processing burden forthe system.
 2. The ultrasound imaging system of claim 1, furthercomprising a display for displaying the image frames, wherein theprocessor is configured to produce a notification on the display thatthe at least one of the image frames in the buffer memory depicts theinjection event.
 3. The ultrasound imaging system of claim 1 wherein theprocessor is configured to include the at least one of image frames thatdepicts the injection event from the buffer memory into a patientrecord.
 4. The ultrasound imaging system of claim 1 wherein theprocessor is further configured to analyze the image frames in thebuffer memory using the machine learning algorithm to detect a motion,wherein the motion is associated with at least one of the needle, thefluid delivered by the needle, and a material removed by the needle. 5.The ultrasound imaging system of claim 1 wherein the injection event isat least one of the needle being positioned at a target location withinthe region of interest and the needle approaching the target locationwithin the region of interest, and wherein the processor is furtherconfigured to analyze the image frames stored in the buffer memory todetect a motion of the needle.
 6. The ultrasound imaging system of claim1 further comprising a display for displaying the image frames, whereinthe processor is configured to automatically mark, in the buffer memory,the at least one of the image frames that likely depicts the injectionevent for inclusion into a patient record in response to theidentifying.
 7. The ultrasound imaging system of claim 1 wherein theinjection event is the injection taking place within the region ofinterest.
 8. The ultrasound imaging system of claim 1 wherein theprocessor is further configured to analyze the image frames stored inthe buffer memory; and to detect a motion of the fluid based on theanalyzing.
 9. The ultrasound imaging system of claim 1 wherein thetransducer is configured to transmit the ultrasound signals in two ormore pulses, the receive circuitry is configured to convert the echosignals into ultrasound image data including a flow information, and theprocessor is configured to analyze the flow information to detect amotion associated with the needle injection procedure, using an opticalflow method that is a Lucas-Kanade method or a Horn-Schunck method. 10.The ultrasound imaging system of claim 1, wherein the processor isconfigured to: capture a flow information for a subset of the echosignals; store the flow information with the image frames in the buffermemory; and to analyze the flow information from the buffer memory todetect a motion; and to identify the at least one of the image framesthat depicts the injection event based on the detected motion from thebuffer memory.
 11. The ultrasound imaging system of claim 1, wherein theprocessor is configured to: determine a difference between two or moreof the image frames in the buffer memory to detect a motion.
 12. Theultrasound imaging system of claim 1, wherein the processor isconfigured to: identify at least one of the image frames that depictsthe injection of the fluid; and mark the at least one of the imageframes for inclusion into a patient record during the needle injectionprocedure.
 13. An ultrasound system for imaging a region of interest ofa subject during a needle injection into the region of interest,comprising: a transducer configured to transmit ultrasound signals toand receive echo signals from the region of interest; a receivecircuitry configured to convert the echo signals into image frames ofultrasound data; a buffer memory configured to store the image frames;and a processor configured to: obtain a first subset of the image framesfrom the buffer memory; compare the first subset of the image frames toeach other; identify, from the first subset, a first image frameindicating that a needle used for a needle injection procedure isapproaching to a target and a second image frame indicating that theneedle has stopped moving at the target based on the comparing; obtain asecond subset of the image frames from the buffer memory, wherein thesecond subset of the image frames have been generated and storedsubsequent to the first subset in the buffer memory; identify, from thesecond subset, a third image frame indicating a tip of the needle usinga machine learning algorithm; obtain a third subset of the image framesdepicting a region around the tip of the needle from the buffer memory,wherein the third subset of the image frames have been generated andstored subsequent to the second subset in the buffer memory; identify,from the third subset, a fourth image frame indicating an injection of afluid from the tip of the needle using a color flow imaging; and storeat least three of the first image frame, the second image frame, thethird image frame and the fourth image frame that depict the needleinjection in a first memory for archival purposes.
 14. The ultrasoundsystem of claim 13 wherein the processor is further configured tocompare the first subset of the image frames stored in the buffer memoryuntil detecting that the needle has stopped.
 15. The ultrasound systemof claim 13 wherein the processor is further configured to analyze thethird subset of the image frames stored in the buffer memory to detectthe fluid around the needle used for the needle injection.
 16. Theultrasound system of claim 13 wherein the processor is furtherconfigured to compare two or more image frames stored in the buffermemory to detect a motion of at least one of the needle used for theneedle injection and the fluid from the needle.
 17. The ultrasoundsystem of claim 13 wherein the processor is further configured toanalyze a flow information associated with the image frames stored inthe buffer memory to detect a motion associated with the needleinjection using an optical flow method that is a Lucas-Kanade method ora Horn-Schunck method.
 18. A method performed by a processor in anultrasound system, the method comprising: receiving ultrasound signalsfrom a region of interest during a needle injection procedure;converting the ultrasound signals into image frames of ultrasound data;storing the image frames in a buffer memory; obtaining a first subset ofthe image frames from the buffer memory; comparing the first subset ofthe image frames to each other; indentifying, from the first subset, afirst image frame indicating that a needle used for a needle injectionprocedure is approaching to a target and a second image frame indicatingthat the needle has stopped moving at the target based on the comparing,obtaining a second subset of the imgage frames from the buffer memory,wherein the second subset of the image frames have been generated andstored subsequent to the first subset in the buffer memory; identifying,from the second subset, a third image frame indicating a tip of theneedle using a machine learning algorithm; obtaining a third subset ofthe image frames depicting a region around the tip of the needle fromthe buffer memory, wherein the third subset of the image frames havebeen generated and stored subsequent to the second subset in the buffermemory, identifying, from the third subset, a fourth image frameindicating an injection of a fluid from the tip of the needle using acolor flow imaging; and saving at least three of the first image frame,the second image frame, the third image frame and the fourth image framethat depict the injection event in a first memory.
 19. The method ofclaim 18, further comprising inserting at least one of the one or moreof the image frames that depict the injection event from the buffermemory into a patient chart.
 20. The method of claim 18 furthercomprising: comparing the first subset of the image frames stored in thebuffer memory to detect a motion of the needle.
 21. The method of claim18 wherein the injection event is a delivery of an injectate to theregion of interest, and wherein the method further comprises: analyzinga flow information in two or more of the image frames from the buffermemory to detect a motion of the injectate, using an optical flow methodthat is a Lucas-Kanade method or a Horn-Schunck method.
 22. The methodof claim 18, further comprising: providing an indication to an operatorthat at least one of the one or more of the image frames from the buffermemory depicts the injection event.